Rethinking the Construction of Effective Metrics for Understanding the Mechanisms of Pretrained Language Models

Pretrained language models are expected to effectively map input text to a set of vectors while preserving the inherent relationships within the text. Consequently, designing a white-box model to compute metrics that reflect the presence of specific internal relations in these vectors has become a common approach for post-hoc interpretability analysis of pretrained language models. However, achieving interpretability in white-box models and ensuring the rigor of metric computation becomes challenging when the source model lacks inherent interpretability. Therefore, in this paper, we discuss striking a balance in this trade-off and propose a novel line to constructing metrics for understanding the mechanisms of pretrained language models. We have specifically designed a family of metrics along this line of investigation, and the model used to compute these metrics is referred to as the tree topological probe. We conducted measurements on BERT-large by using these metrics. Based on the experimental results, we propose a speculation regarding the working mechanism of BERT-like pretrained language models, as well as a strategy for enhancing fine-tuning performance by leveraging the topological probe to improve specific submodules.Comment: Accepted by Findings of EMNLP202

Enhancement of Ride and Directional Performances of Articulated Vehicles via Optimal Frame Steering and Hydro-Pneumatic Suspension

Off-road vehicles employed in agriculture, construction, forestry and mining sectors are known to exhibit comprehensive levels of terrain-induced ride vibration and relatively lower directional stability limits, especially for the articulated frame-steered vehicles (AFSV). The transmitted whole-body vibration (WBV) exposure levels to the human operators generally exceed the safety limits defined in ISO-2631-1 and the European Community guidelines. Moreover, the directional stability limits are generally assessed neglecting the contributions due to terrain roughness and kineto-dynamics of the articulated frame steering (AFS) system. Increasing demand for high load capacity and high-speed off-road vehicles raises greater concerns for both the directional stability limits and WBV exposure. The criterion for acceptable handling and stability limits of such vehicles do not yet exist and need to be established. Furthermore, both directional stability performance and ride vibration characteristics are coupled and pose conflicting vehicle suspension design requirements. This dissertation research focuses on enhancement of ride, and roll- and yaw-plane stability performance measures of frame-steered vehicle via analysis of kineto-dynamics of the AFS system and hydro-pneumatic suspensions. A roll stability performance measure is initially proposed for off-road vehicles considering magnitude and spectral contents of the terrain elevations. The roll dynamics of an off-road vehicle operating on random rough terrains were investigated, where the two terrain-track profiles were synthesized considering coherency between them. It is shown that a measure based on steady-turning root-mean-square lateral acceleration corresponding to the sustained period of unity lateral-load-transfer-ratio prior to the absolute-rollover, could serve as a reliable measure of roll stability of vehicles operating on random rough terrains. The simulation results revealed adverse effects of terrain elevation magnitude on the roll stability, while a relatively higher coherency resulted in lower terrain roll-excitation and thereby higher roll stability. The yaw-plane stability limits of an AFSV are investigated in terms of free yaw-oscillations as well as transient steering characteristics through field measurements and simulations of kineto-dynamics of the AFS system. It was shown that employing hydraulic fluid with higher bulk modulus and increasing the steering arm lengths would yield higher yaw stiffness of the AFS system and thereby higher frequency of yaw-oscillations. Greater leakage flows and viscous seal friction within the AFS system struts caused higher yaw damping coefficient but worsened the steering gain and articulation rate. A design guidance of the AFS system is subsequently proposed. The essential objective measures are further identified considering the AFSV’s yaw oscillation/stability and steering performances, so as to seek an optimal design of the AFS system. For enhancing the ride performance of AFSV, a simple and low cost design of a hydro-pneumatic suspension (HPS) is proposed. The nonlinear stiffness and damping properties of the HPS strut that permits entrapment of gas into the hydraulic oil were characterized experimentally and analytically. The formation of the gas-oil emulsion was studied in the laboratory, and variations in the bulk modulus and mass density of the emulsion were formulated as a function of the gas volume fraction. The model results obtained under different excitations in the 0.1 to 8 Hz frequency range showed reasonably good agreements with the measured stiffness and damping properties of the HPS strut. The results showed that increasing the fluid compressibility causes increase in effective stiffness but considerable reduction in the damping in a highly nonlinear manner. Increasing the gas volume fraction resulted in substantial hysteresis in the force-deflection and force-velocity characteristics of the strut. A three-dimensional AFSV model is subsequently formulated integrating the hydro-mechanical AFS system and a hydro-pneumatic suspension. The HPS is implemented only at the front axle, which supports the driver cabin in order to preserve the roll stability of the vehicle. The validity of the model is illustrated through field measurements on a prototype vehicle. The suspension parameters are selected through design sensitivity analyses and optimization, considering integrated ride vibration, and roll- and yaw-plane stability performance measures. The results suggested that implementation of HPS to the front unit alone could help preserve the directional stability limits compared to the unsuspended prototype vehicle and reduce the ride vibration exposure by nearly 30%. The results of sensitivity analyses revealed that the directional stability performance limits are only slightly affected by the HPS parameters. Further reduction in the ride vibration exposure was attained with the optimal design, irrespective of the payload variations

Hydraulic damping nonlinearity of a compact hydro-pneumatic suspension considering gas-oil emulsion

Hydro-pneumatic suspension (HPS) systems could attenuate broad-frequency-range vibration mainly via the nonlinear hydraulic damping property. While the strut design with shared gas-oil chamber leads to gas-oil emulsion within strut chambers which intricately affects the fluid flows between the coupled chambers and thus the damping force. This study investigated the temperature- and frequency-dependent hydraulic damping properties of a compact hydro-pneumatic suspension strut, in terms of the flow discharge coefficients. A laboratory experiment was performed at nearly-constant strut temperature of 30, 40 and 50 °C, in the frequency range of 0.5-8 Hz. The obtained experimental data are used to identify the discharge coefficients of the emulsion flow across bleed orifices and check valves, which determine the damping property of the considered strut. An analytical model is established, and the simulation results obtained under different strut temperature and excitation frequencies showed reasonably good agreements with the experimental data. The results suggested greater discharge coefficient of the bleed orifice than that of the check valve, which might be due to the relatively complex structure of the check valves. Greater excitation frequency was shown to decrease the discharge coefficients in a nonlinear manner, irrespective of the strut temperature. Greater strut temperature, however, leaded to greater discharge coefficient of the check valve. Increasing the excitation frequency from 0.5 Hz to 8 Hz resulted in nearly 14 % decrease in the discharge coefficient of check valve at a constant strut temperature of 50 °C

ESKNet-An enhanced adaptive selection kernel convolution for breast tumors segmentation

Breast cancer is one of the common cancers that endanger the health of women globally. Accurate target lesion segmentation is essential for early clinical intervention and postoperative follow-up. Recently, many convolutional neural networks (CNNs) have been proposed to segment breast tumors from ultrasound images. However, the complex ultrasound pattern and the variable tumor shape and size bring challenges to the accurate segmentation of the breast lesion. Motivated by the selective kernel convolution, we introduce an enhanced selective kernel convolution for breast tumor segmentation, which integrates multiple feature map region representations and adaptively recalibrates the weights of these feature map regions from the channel and spatial dimensions. This region recalibration strategy enables the network to focus more on high-contributing region features and mitigate the perturbation of less useful regions. Finally, the enhanced selective kernel convolution is integrated into U-net with deep supervision constraints to adaptively capture the robust representation of breast tumors. Extensive experiments with twelve state-of-the-art deep learning segmentation methods on three public breast ultrasound datasets demonstrate that our method has a more competitive segmentation performance in breast ultrasound images.Comment: 12 pages, 8 figure